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(54) IMAGE PROCESSOR AND RECORDING MEDIUM WHERE IMAGE PROCESSING PROGRAM IS 
RECORDED 

(57)Abstract: 

PROBLEM TO BE SOLVED: To obtain the image processor which can 
remove geometric deformation of an obtained image of an image pickup 
object by obtaining the corrected image of the image pickup object 
according to an image of a detected area, a focus position, and the 
position of the detected area. 

SOLUTION: Initialization is performed (SI). Image data are divided 
equally into blocks (S3). An image at a focusing position of each divided 
block is obtained (S5). Obtained distance information is information 
showing contours on a paper surface. Consequently, tilt and swing and 
document distortion information can be obtained. The blocks are tilted 
and connected (S7). Tilt and swing corrections are made (S9). The tilt of 
each block is corrected (S11). The blocks are connected (interpolating 
process) (SI 3). Obtained image data are recorded (SI 5). Consequently, 
geometric corrections can be made without specifying the positions of a 
digital camera and a document. 
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* NOTICES * 

iTPO and NCXPI are not responsible for any 
damages caused by 'the use o£ 'kha.s branslatlon . 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3, In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] By picturizing an image pick-up object in two or more focus locations different, respectively 
with a detection means to detect the field containing the part whose 1st acquisition means which 
acquires two or more images said two or more images are alike, respectively, sets, and the focus suits 
The image processing system equipped with the 2nd acquisition means which acquires the image of said 
amended image pick-up object based on the image of said detected field, said focus location, and said 
detected location of a field. 

[Claim 2] The 3rd acquisition means which acquires the image of one sheet with which the focus suited 
said whole image pick-up object of said acquisition means [ 2nd ] based on the image of said detected 
field, and said detected location of a field. An image processing system including an amendment means 
to amend the image acquired by said 3rd acquisition means based on said focus location and said 
detected location of a field according to claim 1. 

[Claim 3] By picturizing an image pick-up object in two or more focus locations different, respectively 
with the detection step which detects the field containing the 1st acquisition step which acquires two or 
more images, and the part which seiid two or more images are alike, respectively, sets, and is to the point 
The record medium which recorded the image-processing program for making a computer perform the 
2nd acquisition step which acquires the image of said amended image pick-up object based on the image 
of said detected field, said focus location, and said detected location of a field. 
[Claim 4] The 3rd acquisition step which acquires the image of one sheet with which the focus suited 
said whole image pick-up object of said acquisition step [ 2nd ] based on the image of said detected 
field, and said detected location of a field. The record medium which recorded the image-processing 
program containing the amendment step which amends the image acquired by said 3rd acquisition step 
based on said focus location and said detected location of a field according to claim 3. 
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http://www4.ipdLncipi.go.jp/cgi-bin/tran_web_cgi_ejje?u=http%3A%2F%2Fwww4.ipdLn... 7/11/2006 



JP,2000-200344,A [DETAILED DESCRIPTION] 



Page 1 of 4 



* NOTICES * 

JPO and NCIPI are no't responsible for any 
damages caused by "the use o£ "this 'bransla'tlon. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] About the record medium which recorded the image processing system and the 
image-processing program, especially a digital camera is equipped with this invention, and when a book 
etc. is photoed, it relates to the record medium which recorded the image processing system and image- 
processing program which amend distortion and influence of that image. 
[0002] 

[Description of the Prior Art] Drawing 22 is drawing showing the book 2 which is the alphabetic 
character manuscript photoed with a digital camera 1 and a digital camera 1 . When opening and 
photoing a book 2, as shown in drawing 23 , distortion occurs in space. In such a case, if a book 2 is 
photoed as it is with a digital camera 1 , the manuscript image obtained will become like drawing 24 . 
[0003] Furthermore, since the degree of freedom of a digital camera of a camera station is high, as 
shown in drawing 25 , the optical axis of a digital camera 1 and the space of a book 2 are not 
perpendicular [ a digital camera etc. ] in many cases. In such a case, influence arises in the image 
obtained as shown in drawing 26 . 
[0004] 

[Problem(s) to be Solved by the Invention] It may differ and be hard coming to read spacing size and a 
character size according to the geometric deformation of such distortion and influence for every location 
of the picturized alphabetic character. Moreover, when performing character recognition using OCR 
(optical character reader) based on image data, it is easy to produce incorrect recognition of an 
alphabetic character by distortion, influence, etc. Moreover, when the manuscript leans to the direction 
of an optical axis, there is also a problem that a focus suits no fields of a manuscript only by picturizing 
the image of one sheet. Furthermore, the field which is a rectangle configuration originally [, such as an 
alphabetic character field, a photograph field, etc. in a manuscript, ] by distortion or influence in many 
cases may be no longer a rectangle. When p^orming the image-processing approach which chooses the 
suitable compression approach and the amendment approach for every alphabetic character field or 
photograph field, since decision of a field is performed considering a rectangle as a unit in many cases, 
fault may produce it by distortion or influence. 

[0005] Then, this invention aims at offering the record medium which recorded the image processing 
system and image-processing program which can remove geometric deformation of the obtained image. 
[0006] 

[Means for Solving the Problem] In order to attain the above-mentioned piupose, when an aspect of 
affairs with this invention is followed, an image processing system By picturizing an image pick-up 
object in two or more focus locations different, respectively with a detection means to detect the field 
containing the part whose 1 st acquisition means which acquires two or more images two or more images 
are alike, respectively, sets, and the focus suits Based on the image of the detected field, a focus 
location, and the location of the detected field, it has the 2nd acquisition means which acquires the 
image of the amended image pick-up object. 
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[0007] The 2nd acquisition means includes preferably an amendment means amend the image acquired 
by the 3rd acquisition means, based on the 3rd acquisition means which acquires the image of one sheet 
whose whole image pick-up object the focus suited, a focus location, and the location of the detected 
field based on the image of the detected field, and the location of the detected field. 
[0008] The record medium which recorded the image-processing program when other aspects of affairs 
of this invention were followed By picturizing an image pick-up object in two or more focus locations 
different, respectively with the detection step which detects the field containing the 1 st acquisition step 
which acquires two or more images, and the part which two or more images are alike, respectively, sets, 
and is to ihe point A computer is made to perform the 2nd acquisition step which acquires the image of 
the amended image pick-up object based on the image of the detected field, a focus location, and the 
location of the detected field. 

[0009] The 2nd acquisition step contains preferably the amendment step which amends the image 
acquired by the 3rd acquisition step based on the 3rd acquisition step which acquires the image of one 
sheet whose whole image pick-up object the focus suited, a focus location, and the location of the 
detected field based on the image of the detected field, and the location of the detected field. 
[0010] Based on the image of the field detected when these invention was followed, a focus location, 
and the location of the detected field, the image of the amended image pick-up object is acquired. 
[0011] 

[Embodiment of the Invention] Drawing 1 is the perspective view which looked at the digital camera in 
one of the gestalten of operation of this invention firom the fi*ont, and drawing 2 is the perspective view 
(camera back) seen firom back. The digital camera is equipped with the photography carbon button 3, the 
taking-lens section 4, the card insertion hole 5 for inserting card memory, the power button 6, and the 
amendment mode setting switch 7 that sets up the mode which amends distortion and influence when 
picturizing alphabetic character manuscripts, such as a book, with reference to drawing. 
[0012] The photography result by this digital camera is memorized as electric data by the hard disk card 
inside a digital camera (card memory). Card memory is the medium of image data, for example, may use 
a thing like the hard disk card of PCMCIA conformity, and may use a memory card here. Moreover, it 
may replace with such card memory and a mini disc (MD) may be used. Furthermore, you may make it 
transmit image data to a direct printer etc. by a SCSI cable etc., without using card memory as the record 
medium of image data. 

[0013] DraAving 3 is the block diagram showing the configuration of the digital camera in the gestalt of 
this operation. CPUlOl by which a digital camera controls the whole equipment with reference to 
drawing. The CCD section 111 which changes into an electrical signal titie light by which incidence was 
carried out through the taking-lens section 4, An image pick-up object is picturized in RAM 103 which 
memorizes the signal from the CCD section 111 temporarily, and two or more focus locations different, 
respectively. The contrast comparator 105 which chooses the image which the focus suited most by 
comparing the contrast, and is mentioned later and which memorizes a focus location for every block. 
The inclination amendment section 107 which amends by instigating by rotating the whole image based 
on the information on the focus location memorized by the contrast comparator (distance information), 
A local inclination (inclination for every block) is computed to the image with which influence 
amendment was performed, and it consists of the inclination amendment section 109 which performs 
spin compensation, the interpolation section 113 which connects the blocks to which amendment was 
performed, and the card memory section 117 which memorizes the data from the interpolation section 
113. The interpolation section 113 performs processing which smooths during adjoining blocks again. 
Or it changes with a location of which block a pixel value is specifically thought as important in the 
overlap part during (2) blocks which coimects a block in (1) substrate part and connects, the 
corresponding points of (3) blocks are detected and it processes performing lamination processing etc. 
[0014] Drawing 4 is a flow chart which shows the flow of the image processing in a digital camera. 
With reference to drawing, initial setting is performed in step S 1 . Image data is divided into two or more 
blocks at step S3. The image in the focus location of each block is acquired using the contrast method 
later mentioned at step S5. 
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[0015] It connects by leaning each block in step S7, and one image is formed. In step S9, influence 
amendment is perfomied to one image. The inclination of each block is amended in step SI L 
Connection (interpolation processing) of blocks is made in step SI 3. Image data is recorded on the card 
memory section 1 17 at step SI 5. 

[0016] By above-mentioned amendment processing, an image with which ** also photoed space as a flat 
surface perpendicular to an optical axis can be obtained. Moreover, the image which the focus suited as 
a whole can be obtained by connecting the block which the focus suited. Hereafter, the processing 
performed with the flow chart of drawing 4 is explained in detail. 

[0017] [Step SI] initial setting is performed. In addition, the case where the space of a book is picturized 
here is taken for an example, and the case where it is a thing including distortion as image data indicated 
to be to drawing 5 , and influence is assumed. 

[0018] As image data is shown to drawing 6 by the [step] S3 CPU, a division-into-equal-parts rate is 
carried out to two or more blocks. [ 101 ] What is necessary is just to make [ many ] the number of 
blocks that what is necessary is just to lessen when thinking processing speed as important, when 
thinking image quality as important. In drawing 6 , the division-into-equal-parts rate of the image data is 
carried out to the block of 8x8 of 64. 

[0019] The image in the focus location in each block by which [step S5] division was carried out is 
acquired. Specifically, the focus location of a digital camera is first kept away from the location 
sufficiently nearer than the taking-lens section 4 gradually. And it is made for the focus of the image of 
at least one block of 64 blocks to suit. When a focus suits, the distance of Hazama of the mid gear of the 
image of a block and digital camera which the focus suited is measured. And the location and the image 
data of a block of the block whose distance of the focus suited (pixel value) are made into a pair, and are 
memorized. 

[0020] Furthermore, a focus is kept away from the taking-lens section 4, and when a focus suits the 
image of a next different block from last time, the pair of the distance to the image, the location of a 
block, and a pixel value is recorded. Such processing is repeated until the data of all blocks of 64 are 
obtained. 

[0021] Drawing 7 is drawing for explaining the judgment approach of whether the focus of an image 
suited. Drawing 7 (A) If a focus location is changed as shown in - (C), the contrast of an image will 
change. The judgment of whether the focus suited is performed by whether contrast became the highest. 
In drawing 7 , it is judged with the focus location shown in (B) being a focus location (location which 
the focus suited). 

[0022] The distance information acquired by processing of step S5 turns into information which shows 
the contour line of space as shown in dravying 8 . Thereby, the distortion information on influence or a 
manuscript can be acquired. In the gestalt of this operation, the inclination of the image of each block 
can be amended based on this influence and distortion information, and an image as if a photograph was 
taken as a flat surface where space of ** is perpendicular to an optical axis can be obtained by projecting 
on a flat surface perpendicular to an optical axis. Thereby, the readable and high image of the 
recognition rate of OCR can be obtained. In addition, other existing approaches, such as the technique of 
judging most images of a high frequency component as the judgment approach whether the focus is 
correct, in addition to the above-mentioned contrast method to be the image which the focus suited, can 
also be used. 

[0023] Processing which leans a block and is connected is performed in [step S7] step S7. In order to 
simplify explanation, it explains taking the case of a cross section including the block A of dravying 8 . 
That is, drawing which looked at the block of the field which Block A exists in the image data obtained 
as shown in drawing 9 , and is shown by the dotted line from width based on distance information is 
drawing 10 . Each block shown in drawing 10 is leaned so that it may connect with an adjoining block 
smoothly. Block A is leaned as shown by the thick wire part of dravying 1 1 , and specifically, other 
blocks are leaned, as shown by the dotted line. 

[0024] It instigates in [step S9] step S9, and amendment is performed. That is, with reference to drawing 
12 , "A" shows drawing which looked at the block acquired at the process of drawdng 1 1 from the 
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longitudinal direction. Based on the difference HI of the height of the both ends of this connected block, 
the whole block is leaned, as shown in "B." This can compute an angle of rotation based on the 
difference HI of the height of manuscript both ends, and the width of face Wl of a manuscript, and can 
acquire it by performing affine transformation etc. Influence of image data can be amended at this time. 
[0025] The inclination of each block is amended at [step SI 1] step Sll . This is processing which 
specifically makes perpendicular each block of the image data shown by drawing 13 obtained according 
to the process of drawing 12 to an optical axis. For example, an angle of rotation is computed by the 
information H2 on the difference of the height of the edge of the block, and the information W2 on 
width of face to the block A which inclined as shown in "C" of drawing 14 , and it is amended so that 
Block A may become the location of "D" by affine transformation. 

[0026] Processing (interpolation processing) which connects a block in [step SI 3] step SI 3 is 
performed. That is, after processing which amends the inclination of each block in step S 1 1 is 
performed, each block is coimected to one flat surface as shown in drawing 15 . In the processing in step 
SI 1, as shown in drawing 16 , in order to amend the inclination of a block, the block of the shape of a 
rectangle as shown in drawing 17 deforms into the block which is not the rectangle shown in drawing 
18 . Therefore, when connecting two adjoining blocks, a part of block which is not the rectangle shown 
as the continuous line of drawing 19 is started with the rectangle shown by the dotted line, and 
processing connected like drawing 20 is performed. Moreover, in connection, the substrate of an 
adjoining block is detected, it may connect in a substrate part, or the corresponding points of the part 
which overlaps may be recognized and technique, such as making each block rival based on these 
corresponding points etc., may be used. 

[0027] [Step SI 5] The image data obtained by doing in this way is recorded on the card memory section 
117. 

[0028] Geometric amendment can be performed without pinpointing [ in / as mentioned above / the 
gestalt of this operation ] the location of a digital camera or a manuscript. Moreover, the image which 
amendment based on the configuration of the image pick-up object of influence or distortion is not only 
performed, but the focus suited as a whole when the manuscript was distorted in the direction of an 
optical axis can be obtained. 

[0029] Although explained taking the case of the case which is [Modification(s)] where amendment 
processing is performed in a digital camera in the gestalt of above-mentioned operation, you may 
process with a personal computer etc. 

[0030] Drawing 2 1 is a block diagram for explaining, the equipment configuration of the image 
processing system which used the personal computer etc. 

[0031] With reference to drawing, an image processing system consists of CPU201, ROM203 and 
RAM205, a display 207, the floppy disk/CD-ROM drive 209, an input device 211 constituted with a 
keyboard, a mouse, etc., a hard disk 213, and a printer 215. 

[0032] Floppy disk 209a and CD-ROM209b are used to a floppy disk / CD-ROM drive 209. The image 
data set as the object of an image processing is recorded on floppy disk 209a or CD-ROM209b. CPU201 
performs processing shown with llie flow chart of drawing 4 to image data, and the processing result is 
outputted to a display 207 or a printer 215, or it outputs it to floppy disk 209a or CD-ROM209b fiirther. 
[0033] Furthermore, the program which performs processing shown with the flow chart of drawing 4 is 
made to record on record media, such as floppy disk 209a and CD-ROM209b, and it may be made to 
perform an image processing by making CPU201 perform the program. 

[0034] It should be thought that the gestalt of the operation indicated this time is [ no ] instantiation at 
points, and restrictive. The range of this invention is shown by the above-mentioned not explanation but 
claim, and it is meant that all modification in a claim, equal semantics, and within the limits is included. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the perspective view showing the appearance of the digital camera in one of the 
gestalten of operation of this invention. 

[Drawing 2] It is drawing showing the tooth-back section of the camera of drawing 1 . 
[Drawing 3] It is the block diagram showing the equipment configuration of the digital camera of 
drawing 1 . 

[Drawing 4] It is the flow chart which shows the processing which a digital camera performs. 

[Drawing 5] It is drawing showing the example of image data set as the object of an image processing. 

[Dmwing 6] It is drawing for explaining a block division of image data. 

[Drawing 7] It is drawing for explaining the processing which judges a focus condition. 

[Drawing 8] It is the perspective view showing the location of the block which the focus suited. 

[Drawing 9] It is drawing showing the example of image data. 

[Drawing 10] It is drawing showing the cross section of the dotted-line section of drawing 9 . 
[Drawing 11] It is drawing showing the processing which leans a block and is connected. 
[Drawing 12] It is drawing showing the example of influence amendment. 

[Drawing 13] It is drawing for explaining the image data after influence amendment was performed. 
[Drawing 14] It is drawing showing inclination amendment of each block. 
[Drawing 15] It is drawing for explaining connection processing of a block. 

[Drawing 16] It is drawing for explaining deformation of the block by inclination amendment of a block. 
[Drawing 17] It is the top view showing the example of a block. 

[Drawing 18] It is drawing for explaining the condition that the configuration of a block of drawing 17 
deformed by inclination amendment of a block. 

[Drawing 19] It is drawing for explaining coimection processing of the block which deformed. 
[Drawing 20] It is drawing showing two connected blocks. 

[Drawing 21] It is the block diagram showing the configiu*ation of the image processing system which is 
the gestalt of other operations of this invention. 

[Drawing 22] It is drawing showing the relation between a digital camera and a manuscript. 

[Drawing 23] It is drawing for explaining the cause which distortion of a manuscript produces. 

[Drawing 24] It is drawing showing the example of the image data which distortion produced. 

[Drawing 25] It is drawing for explaining the cause which influence produces. 

[Drawing 26] It is drawing showing the example of the image data which influence produced. 

[Description of Notations] 

7 Amendment Mode Setting Switch 

101 CPU 

1 1 1 The CCD Section 

1 05 Contrast Comparator 

107 Inclination Amendment Section 
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109 Inclination Amendment Section 
113 Interpolation Section 
117 Card Memory Section 
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DRAWINGS 



[Drawing 1] 




[Drawing 2] 
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[Drawing 10] 



[Drawing 3] 
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Drawing 5] 



[Drawing 6] 




[Drawing 8] 
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[Drawing 11] 



[Drawing 12] 




[Drawing 13] 



[Drawing 14] 
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Drawing 17] 




[Drawing 18] 
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[Drawing 16] 
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[Drawing 19] 




Drawing 201 




[Drawing 21] 



209a 

lb 



209 



L 




FD/ 
CIHBOH 



209 



205 



SAM 



207 
± 



^4 7s 



J 



CPU 



.201 



211 



.213 



1 



215 



[Drawing 22] 
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